Effect of spatial uncertainty and familiarity on memory for surface colour in natural scenes and Mondrian patterns

How does memory affect surface-colour matching and is there any advantage with familiar scenes? A computercontrolled colour display system was used to present images of natural scenes and Mondrian patterns under two different daylights of correlated colour temperatures 25000 K and 6500 K, each lasting 1 s and separated by an interval lasting 0, 0.1, 0.2, 0.5, 1, 2, or 5 s, with a gradual transition from the first to second image. For natural scenes, one mainly vegetated, the other mainly non-vegetated, the test surface to be matched was a coloured sphere at a fixed location in the scene. For Mondrian patterns, which were either generated afresh from trial to trial or constant throughout the experiment, there was either a single test surface at the centre of the pattern or multiple test surfaces distributed randomly over the pattern. Observers had to decide whether the test surface or surfaces had the same colour over the time course of the stimulus. Performance, measured by a colour-constancy index, was stable with natural scenes and with constant Mondrian patterns, independent of the interval duration. With freshly generated Mondrian patterns, performance was almost constant for intervals up to about 1 s with a central test surface but worsened rapidly with randomly distributed test surfaces. It seems that surface-colour matching is not limited by memory, at least over intervals of seconds, but it is influenced by uncertainty in the spatial organisation of stimuli and the location of the test surface. Introduction Observers can make reliable judgments of surface colour in natural scenes and in artificial geometric displays with similar levels of competence [1-3]. But it is unclear whether these judgements depend differently on memory. For example, it might be expected that the demand for memory in the task would have less impact with familiar natural scenes than with novel geometrical displays and that the predictability of the composition of the scenes might affect performance. To explore these issues, an experiment on surface-colour matching was performed with images of natural scenes and Mondrian patterns under different illuminations with a variable interval between the images being compared. The natural scenes were constant in their layout and location of a test surface throughout the experiment, but the Mondrian patterns were allowed to vary in their spatial novelty and in the number and location of the test surfaces in selected sessions. It was found that the ability of observers to make surfacecolour judgements across increasing intervals depended critically on the predictability of the structure of the stimuli. With natural scenes and constant Mondrian patterns, performance was almost independent of the interval duration, up at least to the maximum of 5 s tested. Methods Apparatus Images of natural scenes were presented on a 20-inch colour CRT display (GDM-F520, SONY, Tokyo, Japan) controlled by a graphics workstation (Fuel, Silicon Graphics inc., Mountain View, CA, USA) with spatial resolution 1600 × 1200 pixels, refresh rate approx. 60 Hz, and intensity resolution 10 bits on each RGB gun. For Mondrian patterns, the display was controlled by an RGB graphics system (VSG 2/5, Cambridge Research Systems Ltd, Rochester, Kent, UK) with spatial resolution 800 × 600 pixels, refresh rate approx. 120 Hz, and intensity resolution 15 bits on each RGB gun. The display system was calibrated regularly with a spectroradiometer (SpectraColorimeter PR-650, Photo Research Inc., Chatsworth, CA, USA) to maintain colorimetric accuracy; errors in a coloured test patch were < 0.005 in (x, y) and < 5% in Y (< 10% at low light levels) in the CIE 1931 (x, y) chromaticity coordinates. Stimuli Two natural scenes, one mainly vegetated, the other mainly non-vegetated scenes were selected from a hyperspectral image database [3]. The images are illustrated in Fig. 1. The test surface was a sphere which was placed physically in the scene and whose surface colour was varied electronically. The images on the screen subtended approx. 18° × 13° visual angle at a viewing distance of 100 cm. The test surface subtended approx. 0.5o. Mondrian patterns consisted of an array of 49 (7 × 7) simulated coloured surfaces, of side 1o visual angle, subtending 7o × 7o as a whole. Each of the surfaces was sampled from the Munsell Book of Color [4]. There was either a single test surface at the centre of the pattern or 25 test surfaces distributed randomly over the pattern. Multiple test surfaces were used to avoid the task becoming a search task, as in [5]. The Mondrian patterns were either generated afresh in each trial or kept constant over all trials for all observers, to allow direct comparison with data from images of the natural scenes. Procedure In each trial, two images of a particular scene were presented in sequence on the CRT display, each for 1 s, in a dark surround and with a variable interval between them. The images differed in their global illuminants. The first was a daylight of correlated colour temperature 25000 K and the second one of 6500 K. These illuminants were selected so as to be compatible with earlier studies in which a surface-colour judgement task was performed with either natural scenes or Mondrian patterns but without the need for memory (e.g. [3]). The interval between the two images lasted 0, 0.1, 0.2, 0.5, 1, 2, or 5 s, during which the first image gradually transformed into the second image (Fig. 2). A gradual transition was used to avoid the generation of transient responses that would occur with the introduction of a blank field. The spectral reflectance of the test surface in the second image was changed by applying a spatially uniform local CGIV 2008 and MCS’08 Final Program and Proceedings 223 illuminant constructed from a linear combination of the daylight spectral basis functions [6]. The chromaticity of this local illuminant was sampled randomly in each trial from a convex gamut in the CIE 1976 (u′, v′) diagram comprising 65 locations. Varying the chromaticity of this local illuminant is closely related to varying the chromaticity of the test surface. Observers had to decide whether the test surface or surfaces had the same surface-colour at the end of the stimulus sequence as at the beginning (i.e. the sequence represented an “illuminant change”) or a different surface-colour (i.e. the sequence represented a “material change”). They signalled their decision using mouse buttons. Each observer performed at least 325 trials for each of the natural scenes and types of Mondrian patterns and for each of the 7 intervals. The gradual transition consisted of the sequential presentation of three intermediate images, which in preliminary observations were found to generate a smooth percept. The global and local illuminants for each of the intermediate images were obtained by linear interpolation between the illuminants of the first and second images in the CIE 1976 (u′, v′) diagram. Observers Eight observers (5 male and 3 female; aged 22–30 years) took part in the experiment. All observers had normal colour vision verified with the Farnsworth-Munsell 100-Hue test, Ishihara pseudoisochromatic plates, Rayleigh and Moreland anomaloscopy and luminance matching, and had normal visual acuity. All observers except one were unaware of the purpose of the experiment. Observers were asked to explain the task back to the experimenter to show that they fully understood. Observers were informed which scene or pattern type and which duration of interval would be tested before starting each experimental sessions, and all observers participated in all experimental conditions. One of the observers was found subsequently unable to make the required judgements, and her data were excluded in the following analysis. Results and Comment The frequency of “illuminant-change” responses in each condition was calculated as a function of the chromaticity of the local illuminant in the CIE 1976 (u′, v′) chromaticity diagram, and the resulting profile was smoothed by applying a loess procedure [7]. From the coordinates of the estimated maximum, a standard colour-constancy index [8] was calculated. Thus, in the CIE 1976 (u′, v′) chromaticity diagram, let c be the distance between the positions of this maximum and the second global illuminant (correlated colour temperature 6500 K), and let d be the distance between the positions of the first and second global illuminants (correlated colour temperatures 25000 K and 6500 K, Figure 2. Time course of stimuli for surface-colour judgements with natural scenes. Over a variable interval (duration 0, 0.1, 0.2, 0.5, 1, 2, or 5 s), the first image was gradually transformed into the second image. This example shows a “material-change” trial with the mainly non-vegetated scene. Figure 1. Images of natural scenes (a) mainly vegetated and (b) mainly non-vegetated. Test surfaces are indicated by arrows. 224 ©2008 Society for Imaging Science and Technology respectively); then the constancy index is 1 − c/d, with 1 representing perfect constancy and 0 perfect inconstancy. Figure 3, a and b, shows the constancy index for each condition, averaged over observers, as a function of the duration of the interval between first and second images. The results in a are for the two natural scenes and the constant Mondrian pattern with a central test surface. The mean constancy index varies little with interval duration, and is consistently higher with the mainly non-vegetated scene (open circles) than with the mainly vegetated scene (solid squares) and constant Mondrian pattern (grey triangles). An advantage for this mainly non-vegetated scene over the mainly vegetated scene has been previously reported when the interval between the images was a dark field lasting up to 5 s [9], and also with a zero interval [3]. The results in Fig. 3b are for freshly generated Mondrian patterns and a central test surface or randomly distributed test surfaces. With a central test surface, the constancy index is approximately constant up to intervals of about 1 s, but then fluctuates. There is also a larger standard error at the longest interval. With randomly distributed test surfaces, the constancy index declines rapidly as the interval increased. A two-way repeated-measures ANOVA (5 scene or pattern types × 7 intervals) was applied to the constancy indices. The main effect of scene or pattern type was significant (F(4,24) = 10.0, p < 0.05) and the main effect of interval and the interaction between scene or pattern type and interval were also significant (F(6,36) = 2.9, p < 0.05, and F(24,144) = 1.7, p < 0.05, respectively). Discussion The ability of observers to make reliable judgments of surface colour in natural scenes and in artificial geometric displays seems to depend little on memory providing that spatial structure is predictable. This contrasts with many tasks involving memory [e.g. 10]. But, when spatial structure and more importantly test-surface location are unpredictable, this ability diminishes. This may be attributable to the loss in a possible transient cue as the interval increased interval [11], so that, if the surface to be matched is unattended, a change in its reflectance may not be detected [5]. Given a constant spatial structure, however, it seems that surface-colour matching is not limited by memory at least over intervals of seconds, and performance may be as good with natural scenes as with geometric displays. Acknowledgements We thank I. Marín-Franch and K. Żychaluk for critically reading the manuscript. This work was supported by the Engineering and Physical Sciences Research Council (grant no. EP/B000257/1).